MULTI-AGENT SYSTEMS MULTI-AGENT SYSTEMS MULTI-AGENT GAMES Rational and Convergent Learning in Stochastic Games

This paper investigates the problem of policy learn-ing in multiagent environments using the stochasticgame framework, which we briefly overview. Weintroduce two properties as desirable for a learningagent when in the presence of other learning agents,namely rationality and convergence. We examineexisting reinforcement learning algorithms accord-ing to these two properties and notice that they failto simultaneously meet both criteria. We then con-tribute a new learning algorithm, WoLF policy hill-climbing, that is based on a simple principle: “learnquickly while losing, slowly while winning.” Thealgorithm is proven to be rational and we presentempirical results for a number of stochastic gamesshowing the algorithm converges.